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DARD-ARS-B 

SUBJECT: "Research on Synthetic Training: Device Evaluation and 
Training Program Development 11 Work Unit SYNTRAIN II 



1. This report presents the results of two studies conducted to evaluate 
a new fixed wing instrument procedures training device and to develop 

a training program for use with the device. 

2. In the first study, the performance of students who received 
synthetic instrument flight training in the device was compared with that 
of students who did not receive synthetic training. The device-trained 
students were~found to be superior during their early training, but this 
advantage tended to decrease as training co~itinued--perhaps because the 
study used the existing training program, only slightly modified, rather 
than a new program developed especially for the device. In addition, 
certain deficiencies noted in the device itself were probably contributing 
factors. The second study concentrated on developing and evaluating a 
training program tailored to the requirements of the new device. Results 
indicated a 40% reduction in the number of flight hours needed to attain 
the twin-engine transition and instrument flight objectives of the course, 
and a considerable decrease in training costs. 

3. This report will be of interest to those concerned with the develop- 
ment and evaluation of training devices and their associated training 
programs. 

FOR THE CHIEF OF RESEARCH AND DEVELOPMENT: 




Colonel, GS 
Chief, Behavioral 
Sciences Office 



HumRRO 
Technical 
Report 
73-20 


- 




i\esearcn on ^ynineuc i raining: 




Device Evaluation and 




Training Program Development 




Paui W. Caro, Robert N. Isley, and 
Oran B. Jclley 




HumRRO Division No. 6 (Aviation) 
Fort Rucker, Alabama 




HUMAN RESOURCES RESEARCH ORGANIZATION 




Work Unit SYNTRAIIM II September 1973 


Approved for 
public release; 
distribution 
unlimited. 


Prepared for 

Office of the Chief of Research and Development 
Department of the Army 
Washington, D.C. 20310 


ERJC 





The Human Resources Research Organization (HumRItO) is a nonprofit 
corporation established in 1969 to conduct research in the field of training 
and education. It is a continuation of The George Washington University 
Human Resources Research Office. HumRRO's general purpose is to improve 
human performance, particularly in organizational settings, through behavioral 
and social science research, development, and consultation. HumRRO's mission 
in work performed under contract with the Department of the Army is to 
conduct research in the fields of training, motivation, and leadership. 



The findings in this report are not to be construed as an official Department 
of the Army position, unless so designated by other authorized documents. 



Published 
September 1973 
by 

HUMAN RESOURCES RESEARCH ORGANIZATION 
300 North Washington Street 
Alexandria. Virginia 22314 

Distributed under the authority of the 
Chief of Research and Development 
Department of the Army 
Washington, D.C. 20310 



FOREWORD 



This report describes an experimental evaluation of an off-the-shelf fixed wing 
instrument procedures trainer and the methodology and concepts employed in the 
development of a new training program for use with the device. The Human Resources 
Research Organization performed this research from 1969 to 1971, under Work Unit 
SYNTRAIN, Modernization of Synthetic Training in Army Aviation. 

An interim report of this activity was delivered to the Aviation School in September 
1969; the present report supersedes the interim report and presents additional findings. 
Related publications in the SYNTRAIN series are; Device-Task Fidelity and Transfer of 
Training: Aircraft Cockpit Procedures Training, HumRRO Technical Report 70-10; An 
Innovative Insf-ument Flight Training Program, HumRRO Professional Paper 16-71; and 
Determining Training Device Requirements in Fixed Wing Aviator Training, HumRRO 
Technical Report 72-11. 

Work Unit SYNTRAIN is a part of the device research program of HumRRO 
Division No. 6 (Aviation) at Fort Ruck r % Alabama. Dr. W 7 allace W. Prophet is Director of 
the Division, and Dr. Paul W. Caro is in charge of training-device research for the 
Division . 

Military support for the study was provided by the U.S. Army Aviation Human 
Research Unit, Fort Rucker, Alabama. LTC Robert O. Carter is the Unit Chief. 

Identification of proprietary products in this report is for purposes of research 
documentation. It does not, in itself, constitute an official endorsement by either 
HumRRO or the Department of the Army. 

HumRRO research for the Department of the Army is conducted under Contract 
No. DAHC 19-73-C 0004. Army Training Research is performed under Army Project 
2Q062107A745. . 



Meredith P. Crawford 
President 

Human Resources Research Organization 



SUMMARY AND CONCLUSIONS 



MILITARY PROBLEM 

At the time of this research, most of the fixed wing synthetic flight training devices 
being used by the Army were obsolete and not optimally suited to existing training 
requirements. In addition, significant advances in aviation training technologies have been 
made that have not yet been applied to Army aviation fixed wing device training 
programs. A HumRRO research program (Work Unit SYNTRAIN) was established to 
(a) identify requirements for fixed wing synthetic flight training devices, (b) develop 
training programs incorporating modern training technology for use with devices procured 
to meet those requirements, and (c) evaluate such programs and devices. 

A device was procured by the Army to meet a particular fixed wing training 
requirement identified during an earlier SYNTRAIN study. An experimental evaluation of 
that device was needed, and was undertaken in SYNTRAIN Sub-Unit II. 



RESEARCH PROBLEM 

Study One of the present research was an experimental determination of transfer of 
training between a new, off-the-shelf, twin-engine, synthetic instrument flight training 
device and the T-42 aircraft in which twin-engine transition and instrument training were 
conducted in the Officer/Warrant Officer Fixed Wing Aviator Course (O/WOFWAC) at 
the U,S, Army Aviation School. Trainees who received synthetic instrument flight 
training in the new twin-engine device were compared with other trainees who did not 
receive synthetic training. The dnference in inflight performance between groups, if any, 
would be indicative of the training contribution of the new device and the synthetic 
training program current at that time. 

Study Two of the research concerned the development and evaluation of a new 
instrument flight training program designed especially for use with the new device. 

In order to provide data that the Army required in conjunction with decisions 
concerning procurement of additional devices, it was necessary to evaluate the device 
before developing an appropriate training program for use with it, The decision could not 
wait for the training program to be developed and tested. Thus, an opportunity was 
provided to determine empirically the value of a training device under two circumstances: 
with a training program suitable for use with an existing device (a common practice when 
new devices are introduced), and with a training program designed specifically for use 
with the new device. 



APPROACH 
Study One 

Prior to the instrument training phase, 20 trainees were randomly selected from the 
rosters of each of the two FY 69 O/WOFWAC classes and assigned to one of two groups. 
One of these groups received no synthetic device training and served as the control group 
for this study. The experimental group received approximately 20 hours of synthetic 
instruction in the new device. The device training that was given to the experimental 
group consisted of a modification of an existing program developed for training devices 
then in use by the Army in the fixed wing training program under study. 



Measures of the relative performanee of the two groups on the end-of -stage cheek 
rides routinely administered to all O/WOFWAC students constituted the major criterion 
for determining transfer of training between the new synthetic device and the aircraft. 
Other criteria employed were attrition from flight training, daily grades during flight 
training, time to check ride, check ride grade, and specifically designed check pilot 
checklists and instructor pilot questionnaires. 

Study Two 

Largely a development activity, Study Two consisted of the administration of 
experimental training packages to small groups of trainees, revision of these packages, and 
the development of a training program based upon experience with them. Deliberate 
effort was made during program development activities to employ modern training 
concepts. The program thus developed was tested experimentally to determine the extent 
to which training with it (in the new device) could be substituted for training in the 
T-42 aircraft. 



RESULTS 
Study One 

Trainees who received training in the new device, when compared with the control 
group trainees, tended to perform in a more satisfactory manner. During early periods of 
training, their attrition rate was lower and they were more likely to be above-average 
students, according to their flight instructors' ratings. During the Stage 1 check rides, 
their performanee of procedural tasks tended to be superior when evaluated subjectively 
by cheek pilots as well as when scored objectively from photographic records of the 
check ride. 

Study Two 

The initial administration of the training program developed in Study Two of this 
research resulted in a \07 f reduction in the number of flight hours required to attain the 
twin-engine transition and instrument flight objectives of the course under study. Use of 
the new program in a similar course produced comparable results. 

CONCLUSIONS 

The following conclusions were reached as a result of this research: 

(1) The 1 new device can contribute to the effectiveness of twin-engine transition 
and instrument training. 

(2) When the device is used with a training program employing modern training 
concepts, significant savings to the Army in inflight training time can be obtained. 

(3) From the transfer-of-training standpoint, the training program used with a 
device is perhaps more important than the device itseif. 

(4) The training concepts employed in the new training program developed 
during this research have application to other flight training courses for both fixed and 
rotary wing aircraft. 
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Research on Synthetic Training 
Device Evaluation and 
Training Program Development 



INTRODUCTION 



Instrument flight training devices, perhaps the most common type of synthetic flight 
training equipment, have played a significant role in military aviation for over three 
decades! The first Link trainer was obtained by the U.S. government in 1934, and 
''instrument trainers" have been used since that time by the Army, Navy, and Air Force 
in fixed wing instrument training courses. Their contribution to flight training efficiency 
is generally recognized. 

The value of any flight training equipment must, however, be determined within the 
context of a training requirement. The mere designation of a device as an instrument 
trainer does not ensure its value in an instrument training program. Its value must rest on 
the suitability of its design for a particular training application and the appropriateness of 
the manner in which it is used (1, 2J. When training requirements change, as they often 
do when missions change or when new aircraft enter the inventory, the synthetic training 
devices in use and their associated training programs must be reevaluated to assure their 
continued effectiveness. 

During FY 67, significant changes were made in flight training programs at the U.S. 
Army Aviation School. One of these was the adoption of the T-42 (Baron) as the 
principal instrument training aircraft (Figure 1). Jt'he use of a new instrument training 
aircraft, particularly since it was a twin- instead of a single-engine aircraft, made it 
necessary to revise objectives of all instrument training programs to include, among other 
things, objectives related to twin-engine transition training. 

T-42 (Baron) Aircraft 




At that time, the Army's most modern fixed wing instrument training device was 
Device 2B12A, which is shown in Figure 2. The training requirement, which resulted in 
the ' 'ntual development of the 2B12A, was stated originally in 1957, and the proposed 
device was intended for use during the 1960-1965 time frame. Device 2B12A has no 
twin-engine simulation capability and no cockpit motion. Its aerodynamics and engine 
simulation are limited, and its cockpit and sound characteristics do not correspond, even 
approximately, to those o£. the new instrument training aircraft. In spite of these 
limitations, Device 2B12A was the best available for synthetic instrument training at the 
Aviation School. 



Also during FY 67, the Army added the U-21 aircraft to its inventory, and a U-21 
pilot transition training program was instituted at the Aviation School shortly thereafter. 
No existing Army training devices were suitable for use *or U-21 pilot training, and a 
device was not procured with the aircraft. Consequently, it was necessary, at least 
temporarily, to conduct all U-21 transition training in the aircraft itself. 

The Aviation School, in August 1967, expressed concern over the efficiency of its 
fixed wing training capability with respect to the introduction of these new aircraft. 
Aviation School personnel expressed the belief that training devices offer great potential 
for reducing training costs r id upgrading the quality of flight instruction, HumRRO was 
asked to review the existing fixed wing training programs, and to identify areas where 
training might be improved through more efficient use of synthetic training equipment. 

The requested study was undertaken during the second quarter of FY 68, and its 
conclusions have been reported in a HumRRO report, Determining Training Device 
Requirements in Fixed Wing Aviator Training (3). The present report describe* additional 
work requested by the Aviation School as a result of conclusions of the earlier study 
concerning the need for new synthetic training equipment for use in conjunction with 
instrument training in the T-42 aircraft. 



Instrument Panal of 2B12A Trainer 




Figure 2 
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The conclusions of the earlier study concerning instrument training were based upon 
a detailed review of the requirement for instrument training, the features of the newly 
introduced training aircraft, and the characteristics of existing military as well as com- 
mercially available training devices. During that review, the assumed inadequacies of 
Device 2B12A were confirmed, and information was developed about a soon-to-be- 
available commercial device that appeared to have potential for use in twin-engine 
transition and instrument flight training. Subsequently, the Army contracted with the 
manufacturer to acquire two of these devices 1 for evaluation, and requested HumRRO to 
provide technical assistance in a determination of their training value to the Army and to 
develop training programs for use with them. (Interior and exterior views of the new 
device are shown in Figures 3 and 4.) 

This report describes HumRRO's activities in response to the Aviation School's 
request for assistance in an evaluation of the new device, and for development of training 
programs for use with it. The Aviation School requested that development of training 
programs not delay an initial evaluation of the device, however, since the evaluative data 
were needed in connection with a decision concerning procurement of additional copies 
of the device. Consequently, two studies were undertaken. In the first, a transfer-of- 
training experiment was conducted to provide data for use by the Aviation School in its 
decisions concerning procurement of additional devices. 

The second study consisted of a series of activities leading to development of a 
training program for use with the device. It should be noted that Studies One and Two 
are relatively independent. Completion of Study One was not a prerequisite to 
Study Two, although the familiarity with the equipment gained during Study One was of 
considerable Value during the subsequent work. 

At the time of this research, the Officer/Warrant Officer Fixed Wing Aviator Course 
(0/WOFWAC) at the Aviation School was graduating approximately 700 aviators each 
year. 2 This course was the Army's largest fixed wing training course, and it was the 
intent of the Aviation School to use the new device in that course. Consequently, that 
course became the setting for Study One and for most of Study Two. 

The O/WOFWAC consisted of a four-phase flight training program. Phases I and II 
totaled approximately 100 hours of dual instruction and solo practice of basic and 
advanced contact flight training. This training was conducted at Fort Stewart. Georgia, in 
the T-41 aircraft. 

Upon successful completion of Phases I and II, the trainee was transferred to the 
Aviation School at Fort Rucker, Alabama, where he received the third and fourth phases 
of O/WOFWAC training. Phase III, Twin-Engine Qualification and Instrument Training, 
lasted for eight weeks, and consisted of 60 hours of flight training in the T-42A aircraft 
and approximately 100 hours of academic, (i.e., classroom) instruction, including 
21-24 hours of synthetic instrument training in Device 2B12A. 3 The purpose of Phase III 
was to train enrollees for the award of an Army Instrument Card (Standard) in accord- 
ance with Federal Aviation Administration standards and applicable Army regulations, 
and to qualify the students in the twin-engine airplane. 

l The new device referred to in this research is the General Aviation Trainer Model 2 (GAT-2) 
manufactured by Simulation Products Division, Singer Corporation. It is identified for research 
documentation purposes only, and its use or citation does not constitute an official endorsement or 
approval by either the Human Resources Research Organization or the Department of the Army. (Since 
this report was prepared, the new device has been type-classified by the Army and designated Device 2B30.) 

2 Further student inputs to the O/WOFWAC were suspended in October 1970. 

3 In preparation for the instrument phase, trainees received 12 hours of synthetic training in 
Device 1-CA-l during Phase I and Phase II of the O/WOFWAC. This training was restricted to attitude 
q instrument flying; no navigation procedures were included in the course. 

ERIC 
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Interior— The New Device 




Phase III flight, academic, and synthetic training was conducted concurrently over 
the eight-week period by a civilian contractor. The 60 flight hours were divided into 
10 hours of twin-engine transition training and 50 hours of dual instruction in instrument 
flying techniques. Two check rides were given in the Phase: the first, Stage 1, normally 
occurred during the fifth week of training (25-30 flight-hour level), and the second, 
Stage 2, occurred during the eighth week (55-60 flight-hour level) of training. Both check 
rides were conducted by check pilots assigned to a flight examiner group whose primary 
function was to evaluate the training provided by the civilian contractor. 

Phase IV, Contact and Tactical Employment, was an eight-week phase in which 
students were trained in the use of Army fixed wing aircraft under tactical conditions 
and in the tactic -s employment of Army aviation. Phase IV flight training was conducted 
in the 0-1 "Bird Dog" airplane. Graduates of this phase were awarded aviator wings and 
were typically assigned either to operational units or to graduate level aviator courses at 
the Aviation School. Additional descriptive information concerning each phase of 
O/WOFWAC training has been reported by Caro et al (3). 



Study One 



EVALUATION OF A TRAINING DEVICE USING 
AN EXISTING TRAIN INo PROGRAM 

RESEARCH OBJECTIVE 

This section of the report describes a transfer of training experiment conducted to 
determine the suitability of the GAT-2 for Army training. Information concerning the 
GAT-2 was developed during the earlier survey of the Aviation School's synthetic training 
device requirements (3). Information concerning the device suggested, on analytical 
grounds, that appropriate use of the GAT-2 would meet some of the Aviation School's 
requirement for a device for fixed wing instrument training. However, an empirical 
evaluation was needed to verify the new equipment's suitability for use in an operational 
flight-training program. Because of a need to make a decision as soon as possible 
concerning procurement of additional devices, time did not permit the development of a 
training program that would permit optimum use of the device. Consequently, it was 
necessary to adapt a training program being used with the 2B12A device for use during 
the evaluation of the GAT-2. 

The objective of the suitability evaluation was to provide, as rapidly as possible, data 
for use by the Aviation School in its decisions concerning procurement of additional 
devices. The data were generated through the conduct of a transfer-of-training experiment 
that compared the flight performance of students trained in the new device using an 
existing training program with that of students who received no synthetic training. 



METHOD 

SUBJECTS 

The trainees participating in this study were officers, warrant officers, or warrant 
officer candidates enrolled in two FY 69 O/WOFWAC classes (Classes 69-15/16 and 
69-19/20). Prior to the beginning of Phase III, 20 trainees were randomly selected from 
the roster of each class and assigned to one of two groups, for a total of 40 trainees. The 
only restrictions placed on the selection procedure were that (a) trainees not have more 
than 50 hours flight experience prior to entering Phase I of the O/WOFWAC, and 
(b) trainees not have been recycled during their Phase I and Phase II flight training at 
Fort Stewart. 

The remaining members of each class, excluding foreign nationals, were categorized 
on the basis of their prior aeronautical experience levels. Students with 50 hours or less 
were administratively tagged as low time students, while those with more than 50 hours 
were tagged as high time students. These latter students were not manipulated in any 
way, and the research interest in them was not revealed to Aviation School personnel 
during the course of the study. 



The number of students in each group and their respective mean prior flight 
experience levels are shown in Table 1. The low prior time group had a mean of 
approximately six hours; the high prior time group had a mean of approximately 
240 hours. Differences in mean prior experience between the Experimental, Control, and 
Low Time Groups in Table 1 do not appear to be significant. 

Table 1 



Number of Students in Each Group and 
Mean Prior Experience Levels 







Mean Prior 








Experience 


Range 


Group 


N 


(Hours) 


(Hours) 


Experimental 


24 


6.8 


0-44 


Control 


16 


10.6 


0-50 


Low Time 


20 


6.5 


0-50 


High Time 


37 


240.6 


51-900 



EXPERIMENTAL DESIGN 

A simple, one-way analysis of variance model was used in the design of this research, 
with group assignment as the experimental variable. Where frequency data were used, Chi 
Square data analysis techniques were employed. 

Groups Involved 

The Experimental Group, consisting of 12 trainees from each of the two 
O/WOFWAC classes, received synthetic training in the new device. The Control Group, 
the size of which was restricted to levels that the civilian flight contractor could handle 
in the event such students might require additional training, consisted of eight trainees 
from each class who received no synthetic training. 

No changes were made in the flight or academic portions of training for these two 
groups. The Low Time and High Time Groups both received the normal O/WOFWAC 
program of instruction, including training in Device 2B12A, and were distinguished only 
by the differences in prior experience levels as indicated in Table 1. 

Two classes were used in this research because of the relatively small number of 
Experimental and Control Subjects who could be administratively handled in a single 
O/WOFWAC class. Prior research has often shown significant class differences because of 
the use of different groups of instructional personnel; however, in this case, the same 
group of instructors was used throughout and there were no other systematic changes in 
group treatment between classes. 

Synthetic Training in the New Device 

Conventional synthetic training in the O/WOFWAC occurs during the second 
through the fifth weeks and is concurrent with inflight instruction. The synthetic training 

O 
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administered to the Experimental Group in the new device differed from that conven- 
tionally administered to O/WOFWAC trainees in three major aspects. First, the training 
began one week prior to flight line instruction. 4 Second, primary emphasis during this 
early period was on normal and emergency procedural tasks common to light twin-engine 
aircraft and capable of simulation in the GAT-2. This procedural training was intended to 
assist the trainee during the transition portion (i.e., the first 10 hours) of T-42 flight 
training. The third difference was that, in addition to pilot training in the device, each 
student received an equal amount of time performing copilot tasks and/or observing a 
student acting as pilot. (A determination of the effects of providing copilot training was 
not directly addressed in this study; it is assumed to have been of some value, particu- 
larly during the initial learning of both twin-engine and instrument procedures.) 

Except for differences dictated by the unique operating characteristics of the new 
device, the remainder of the Experimental Group's synthetic training was identical in 
content to that conventionally given to O/WOFWAC trainees in Device 2B12A (and to 
the Low Time and High Time subjects identified in Table 1). The syllabi used with both 
devices are shown in Appendix A. All synthetic and flight instruction was provided by 
the civilian contractor. The content and sequence of the flight training was independent 
of Experimental or Control Group assignment. 

PERFORMANCE EVALUATION 

To assess the training value of the new device, flight performance of trainees in the 
four groups was compared. Primary interest centered on flight performance of the 
Experimental and Control Groups, The High Time and Low Time 2B12A trained groups 
were included in some analyses (where data were available) to provide additional 
information; however, no direct attempt was made in this research to compare the 
relative training effectiveness of the new device and the 2B12A. 5 

Flight performance data were extracted from four sources: photographic recordings 
of aircraft performance during the two instrument-phase check rides; recordings of 
communications between each trainee, check pilot, and ground controller during the two 
check rides; check pilot checklists and instructor-pilot questionnaires developed by the 
research staff; and trainee flight records. 

Photographic Records 

Photographic records of the two check rides were obtained through the use of a 
16mm motion picture camera, mounted on the right rear seat rails of the T-42 aircraft 



4 The Experimental Group started synthetic training during the administrative week of training, a 
period normally reserved for inprocessing and related administrative activities. Inprocessing of the 
Experimental Group was accomplished on a special schedule utilizing the haU day not required for 
synthetic training. 

5 A comparison of the relative training value of the new device and the 2B12A was beyond the 
scope of this research. Equipment limitations precluded obtaining photographic data on the 2B12A 
trained groups. Nevertheless, performance measures that were readily available from trainee flight records 
and instructor pilot questionnaires of the 2Bl2A-trained groups were included in the appropriate 
analyses to provide information about trainees undergoing the routine program of instruction. 

o 
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(Figure 5). Standardized check rides were administered so that all trainee performance 
would be recorded and filmed on the same series of tasks performed in the same 
sequence. A description of the maneuvers included in each check ride is shown in 
Appendix B. 



Camera and I ntervalometer Mounted in T-42 Aircraft 




Figure 5 



Time-lapse photographic techniques were employed, and the films were scored after 
completion of the check ride by a team of nonrated enlisted men working under the 
supervision of a qualified instructor pilot. Photographic and scoring procedures used were 
developed during earlier HumRRO research (4, 5). 

Communication Recordings 

Portable cassette-type tape recorders were used to record transmissions between the 
aircraft and Air Traffic Control (ATC) ground controllers and communications between 
the check pilot and the trainee on the aircraft intercom system. These recordings 
provided information concerning the trainees' communication skills, as well as 
information related to deviations of each check ride from the standardized flight profile 
caused by air traffic, weather, or other uncontrollable events, the latter information being 
of value during performance scoring activities. 

Checklists and Questionnaires 

A checklist was completed at the conclusion of each check ride. The checklist 
required the check pilot to rate the trainee's performance on a number of maneuvers and 
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procedures that did not lend themselves to evaluation by photographic means. The 
checklists are shown in Appendix C, 

A questionnaire was developed for use with instructor pilot personnel employed ^y 
the flight training contractor. The questionnaire, designed for administration at the end 
of the first week of flight training, asked the instructor pilots to rate their trainees' 
progress on a number of procedural and aircraft control tasks that had been introduced 
in the hew device prior to the trainees' arrival on the flight line. The ratings consisted of 
instructor pilot determinations of whether the trainee's performance in the aircraft during 
his first week of training was generally above average, average, or below average. 

Trainee Flight Records 

In addition to the specially developed data sources just described, information 
concerning trainee attrition rates, daily flight grades, check-ride grades, and time to check 
ride was extracted from the flight records of all trainees of interest routinely maintained 
by the Aviation School. 



RESULTS 



DATA FROM PHOTOGRAPHED CHECK RIDE MANEUVERS 6 



The photographed portion of the fifth week check ride (Stage 1) consisted of a Very 
High Frequency Omnidirectional Range (VOR) navigation problem conducted under 
simulated instrument flight conditions, 7 With the check pilot acting as Air Traffic 
Control (ATC), each trainee was issued a standardized clearance. To comply with this 
clearance, each student had to: orient himself in relation to a VOR facility; select and fly 
an appropriate interception course to a specific radial; intercept and track inbound on 
that radial; hold at the VOR until cleared for an approach; execute the published 
approach when cleared; and,, at minimum altitude, execute a missed approach. 

The photographic record of trainee performance of this task was scored in the 
manner reported elsewhere (5), For scoring purposes, 16 performance measures were 
derived. Ten of these measures reflected the trainee's ability to control the aircraft within 
acceptable tolerances, and six performance measures reflected the trainee's ability to 
follow the standard procedures involved in complying with the ATC clearance. The 16 
measures and the criteria or tolerances associated with them are identified in Table 2. 

Error /no error frequency counts were made for each group, and Chi Square analyses 
were 'performed on each of the 16 performance measures identified in Table 2, The 



6 Photographic records of the check rides were obtained only on Experimental and Control Group 
trainees, 

7 The camera system also operated during other portions of the check ride; however, the IFR 
navigation problem was of primary concern during this check ride. 
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Table 2 



Performance Measures Derived From Stage 1 
Check Ride VOR Navigation Problem 



Performance Measure 


Type of Item 


Criterion or Tolerance 


Number 


Description 


(1 \ 


VOR Oripntatinn 
v \*f i i \J i lc 1 1 La 1 1 u 1 1 


Procedural 


Proper/Improper 




foiircp Qp|pr % tinn 


Procedural 


Proper/Improper 


(3) 


1 1 aLI\ Ml LCI L»C }J L I U 1 1 


Procedural 


Proper/Improper 




Inbound Track.' 






(4) 


Altitude 


A/C Control 


± 100 feet 


(5) 


Airspeed 


A/C Control 


± 10 knots 


(6) 


Ground Track 


A/C Control 


± 5 degrees 


(7) 


Procedures 


Procedural 


Proper/Improper 




Holding Pattern: 






(8) 


Altitude 


A/C Control 


± 100 feet 


(9) 


Airspeed 


A/C Control 


± 10 knots 


(10) 


Heading 


A/C Control 


± 10 degrees 


(11) 


Time Outbound 


Procedural 


Proper/Improper 


(12) 


Procedures 


Procedural 


Proper/Improper 




Approach: 






(13) 


Procedure Turn 








Altitude 


A/C Control 


+ 1 00 feet 


. (14) 


Inbound Final Track 


A/C Control 


±10 degrees 


(15) 


Low Station Altitude 


A/C Control 


+ 100 feet 


(16) 


Altitude at Missed 








Approach 


A/C Control 


+ 100 feet 



frequency of errors for each group and the obtained Chi Square for each of the 16 
measures are shown in Table 3. 

Differences between groups (p < .05) were found for 4 of the 16 measures. In each 
case, the Experimental Group made fewer errors than did the Control Group. The 
variations in numbers reported in Table 3 reflect differences in the number of students 
completing various portions of the check ride, as well as differences in the scor ability of 
filmed segments. Other data losses occurred due to attrition prior to the check ride, 
camera system malfunctions, and aircraft breakdowns. The data in Table 3 are based on 
examination of 16 Experimental Group films and 13 Control Group films. Except for 
attrition, which is discussed later in this report, the data losses appear to have occurred 
at random and are not considered to have had any systematic effect on the results 
reported here. 

The eighth week or end-of-phase (Stage 2) check ride was in the form of a'simulated 
instrument flight that required the student to file an IFR flight plan; make an instrument 
take-off; follow a radar vector to course interception; intercept and track outbound on a 
specific radial; intercept and track inbound on a specific Victor airway; hold at his 
clearance limit (an intersection along the airway); upon receiving further clearance, 
proceed along the airway to his destination facility, hold at the facility until cleared for 
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Table 3 



Error Frequency by Group and Obtained Chi Squares for 
16 Flight Performance Measures 



Performance Measure 


Group Enor Frequency 


Chi 

Square 3 
I 


Number 


Description 


Experimental 


Control 


Errors 


No Errors 


Errors 


No Errors 


N 


% 


N 


N 


% 


N 


/ 1 \ 


\/OR Or icin tatinn 

vun vjneniaiion 


3 


20 


12 


3 


23 


10 


0.039 




Prtl ir CO Qolontinn 

uuui be ocicciion 


3 


20 


12 


8 


62 


5 


5.038* 


(3) 


Track Interception 


3 


20 


12 


8 


62 


5 


5.038* 




Inbound Track: 
















(4) 


Altitude 


4 


27 


11 


2 


18 


9 


0.257 


(5) 


Airspeed 


1 


7 


14 


0 


0 


11 


0.762 


(6) 


Ground Track 


3 


20 


12 


6 


60 


4 


4.167* 


(7) 


Procedures 


4 


27 


11 


4 


35 


7 


0,280 




Holding Pattern: 
















(8) 


Altitude 


6 


38 


10 


4 


33 


8 


0.052 


(9) 


Airspeed 


0 


0 


16 


1 


8 


11 


1.383 


(10) 


Heading 


C 


0 


16 


1 


8 


11 


1.383 


(11) 


Time Outbound 


2 


12 


14 


2 


17 


10 


0.097 


(12) 


Procedures 


2 


12 


14 


7 


58 


5 


6.604* 




Approach: 
















(13) 


Proceduie Turn Altitude 


0 


0 


15 


0 


0 


10 


N/A 


(14) 


Inbound Final Track 


6 


40 


9 


3 


30 


7 


0.260 


(15) 


Low Station Altitude 


1 


7 


14 


2 


20 


8 


1.010 


(16) 


Altitude at Missed 


















Approach 


4 


27 


11 


2 


20 


8 


0.146 



^he symbol * indicates statistical significance (p < .05. di-\). 



an approach; execute the published VOR approach when cleared; at minimum altitude 
execute a missed approach; request clearance to his alternate destination; follow a radar 
vector to the LOM (Locator Outer Marker) serving his alternate destination; and, when 
cleared, execute an instrument landing system (ILS) approach. 

Filming of this check ride was hampered by inclement weather to the extent that 
the simulated instrument flight just described could not be executed in its entirety and 
only the ILS approach was considered sufficiently standardized for evaluation using the 
photographic scoring techniques. 8 In addition, attrition in the Control Group prior to 
this check ride markedly reduced the sample size. Nevertheless, data from 18 ILS 



8 The majority of these check rides were fJown under actual instrument conditions, and flight 
routes could not be standardized to the extent necessary for research purposes. Approximately one-half 
the flights were terminated with an ILS approach. 



approaches (12 Experimental and 6 Control) were obtained. Scoring for the ILS approach 
began at glide slope interception and was terminated at either published altitude 
minimums or when the check pilot removed the trainee's hood, whichever occurred first. 
Normally, this portion of the ILS approach required about 3 to 3 1/2 minutes. The total 
time each trainee was outside the "doughnut" on the ILS indicator during this period 
was extracted from the photographic records. (Being outside the doughnut indicated a 
course deviation of at least one-half degree.) 

The mean frequency and mean percent of time off course for each group are shown 
in Table 4. Although the frequency off -course means for the two groups are similar, the 
time off-course means indicate that the Control Group was off course during a larger 
portion of the approach than the Experimental Group. This difference, however, is not 
significant. 

Table 4 

Mean Frequency Off Course and Mean 
Percent Time Off Course on ILS 







Mean Frequency 


Mean Percent Time 


Group 


N 


Off Course 


Off Course 3 


Experimental 


12 


3.67 


29 


Control 


6 


3.83 


38 



a Time off course reflects deviation from both course and ylide slope. 



DATA FROM COMMUNICATION RECORDINGS 9 

The primary intent of the use of tape recorders during the Stage 1 and Stage 2 
check rides was to provide data of value to research personnel scoring the check ride 
films. However, an attempt was made to use the course corrections given by the final 
controller during the Ground Controlled Approach (GCA), (PAR) on the Stage 1 check 
ride, as an indication of trainee proficiency with this type of approach. The number of 
times the final controller reported the trainee off course and/or off glide slope was 
extracted from each tape. It was found, however, that variations among ground con- 
trollers were wide enough to make the reported deviations suspect as trainee performance 
data. The recordings proved useful as debriefing aids for the check pilots, and a number 
of check pilots recommended that recorders be used routinely on all check rides because 
of their value in debriefing. 

CHECKLIST DATA 1 0 

From the checklists completed by the check pilots at the end of each Stage 1 check 
ride, subjective ratings were obtained of trainee performance during four flight tasks that 
were not included in that portion of the check ride that was scored from photographs. 
These four maneuvers could not be standardized to the extent necessary for after-the- 
event evaluation by means of photographic record. They either allowed essentially 
nonstandard performance (e.g., Recovery from Unusual Attitudes), or they involved 
stimuli outside the aircraft that could not be photographed (e.g., VFR Landing). 



* Communications data were available only on Experimental and Control Group trainees. 
1 0 Checklist data were* obtained on Experimental and Control Groups only. 



These tasks were a VFR Take-off, Recovery from Unusual Attitudes, Single Engine 
Procedures, and a VFR Traffic Pattern and Landing. The ratings consisted of cheek pilot 
determination of whether trainee performance on each maneuver was "proper" or 
"improper," The relative performance of the Experimental and Control Groups on each 
rated task was evaluated using the Chi Square technique. Table 5 shows the number of 
trainees in each group rated proper or improper in the performance of each task. A 
significant difference between groups was found for the single-engine procedures 
task (p < .05). 

Table 5 



Check Pilots' Ratings of Stage 1 Maneuvers 







Rated 


Voper 


Rated Improper 


Chi 
Square 3 


Task 


Group 


N 


h 


N 


VFR Take Off 


Experimental 
Control 


20 
11 


87 
73 


3 
4 


1.121 


Recovery from 
Unusual Attitudes 


Experimental 
Control 


16 
12 


70 
80 


7 
3 


510 


Single Engine 
Procedures 


Experimental 
Control 


15 
4 


65 
29 


8 
10 


4.710* 


Traffic Pattern 

and Landing (VFR) 


Experimental 
Control 


9 

5 


39 
42 


14 

7 


.021 



a The symbol * indicates statistical significance [p < ,05; df^'l). 



The checklist for the Stage 2 eheek ricle provided subjective ratings on seven flight 
tasks. These included an Instrument Take-Off, Track Interception, Holding Pattern, 
Missed Approach, Communications, Partial Panel, and Single-Engine Procedures. Chi 
Square analyses of these ratings failed to yield significant differences at the .05 level 
between groups on any of these tasks. 



INSTRUCTOR PILOT QUESTIONNAIRES 

The instructor pilots employed by the flight training contractor were surveyed at the 
end of the first week of flight training. The survey included the instructor pilots of all 
four groups of trainees and asked them to rate their trainees' progress on a number of 
procedural and aircraft control tasks. These tasks had been introduced to the experi- 
mental trainees in the new device during the period prior to their first aircraft flight. 
Regardless of group assignment, all trainees received training on these tasks in the aircraft 
during the first week of flight training. The ratings consisted of instructor pilot deter- 
mination of whether trainee performance in the aircraft during the first week of training 
was generally above average, average, or below average. 

It should be noted that synthetic training in the instrument phase of O/WOFWAC 
normally does not begin until the second week of the course. Consequently, the Control 
Group had not missed any synthetic training normally administered to O/WOFWAC 
trainees at the time this instructor pilot questionnaire was administered. The Control 
Group, then, was similar in treatment a\ that point to other members of their 
O/WOFWAC class who met the same selection criteria (i.e., minimum prior aeronautical 
experience) and, in fact, had been selected randomly from among such trainees. 
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In order to increase the sensitivity of a statistical test of the effectiveness of the 
training received by the Experimental Group, the Control Group of 16 was combined 
with 19 additional low prior experience trainees in their classes (on whom questionnaire 
data also had been collected), and the differences in instructor pilot ratings of the 
Experimental Group and the thus augmented Control Group were investigated. These 
data are indicated in Table 6. 

Table 6 



Instructor Pilot Ratings of First Week's Performance 



Group 


... 

Instructor Pilot Ratings 


Percent 
Above 
Average 


Chi 
Square 


Above Average 


Average or Below 


N 


% 


N 


Experimental 


24 


10 


14 


42 












5.630 


Augmented Control 


35 


5 


30 


14 




High Time 


35 


11 


24 


31 





The different frequencies of ratings between the Experimental and the augmented 
Control Group w^re tested for statistical significance using the Chi Square technique. The 
greater frequency of above-average ratings obtained by the Experimental Group was 
found to be significant (p < .05, X 2 = 5.630, elf = 1) when compared with the augmented 
Control Group. 

As might be expected, flight instructors quickly discovered, without being told, 
when a student had had significant amounts of prior aeronautical experience. Such 
students typically excel, especially early in training. It is therefore interesting to note in 
Table 6 that the first week instructor ratings of the Experimental Group are somewhat 
better than those received by the High Prior Time Group, the group most likely to excel 
at this stage of training. Although this difference is not significant, it is noteworthy that 
the Experimental Group was performing at a level approximately equal to that of 
students with relatively large amounts of prior aeronautical experience. 

DATA FROM TRAINEE FLIGHT RECORDS 
Attrition 

Table 7 indicates the attrition (eliminations and recycles due to flight skill defi- 
ciencies) in terms of the number of students who failed to complete the phase with their 
class for reasons of flight deficiency. (Two trainees in the High Prior Time Group were 
attrited for reasons other than flight deficiency.) Five of the Control subjects were unable 
to complete Stage 1 training with their class, while only one of the Experimental subjects 
attrited at this stage. A Fisher Exact Probability Test of this difference was significant 
(p < .05). Similar analysis of the Stage 2 data failed to yield a significant difference. 
Attrition figures for the low prior time and high prior time groups are also included in 
Table 7. It should be noted that significant differences in Stage 1 attrition were also 
found between the Control Group and the Low Time Group and between the Control 
Group and the High Time Group. These findings are not unexpected and will be 
addressed in the discussion section. 
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Table 7 



Flight Deficiency Attrition 



Group 


Number Entering 
Training 


Attrition 


Stage One 


Stage Two 


Total 


Experimental 


24 


1 


1 


2 


Control 


16 


5 


0 


5 


Low Time 


20 


1 


0 


1 


High Time 


37 


2 


1 


3 



Time to Check Ride 

The flight times required for students to pass the Stage 1 and Stage 2 check rides 
were extracted from individual flight records and are summarized in Table 8 where it can 
be seen that there was very little difference between groups. Such differences as did exist 
were found not to be significant using analysis of variance techniques. 

Table 8 

Flight Hours to Pass Check Ride for 
Each Group, by Stage of Training 



Group 




C-1 Stage 




C-2 Stage 


Mean 


N 


SD 


Mean 


N 


SD 


Experimental 


30.1 


23 


1.70 


58.8 


22 


2.58 


Control 


30.3 


1 1 


2.12 


58.6 


11 


3.71 


Low Time 


29.9 


19 


1.31 


58.8 


19 


2.85 


High Time 


29.3 


33 


1.12 


57.0 


32 


2.47 



Instructor* Assigned Daily Grades During Training 

Instructor-assigned daily flight grades wer-a extracted from the flight records of each 
student. These grades are subjective in nature and are assigned in accordance with the 
Uniform Flight Grading System/ 1 To facilitate statistical analysis, daily grades of U, C, 
B, and A were assigned the numerical values of 1, 2, 3, and 4, respectively. The total 
number of graded flights for each trainee during ea^h stage of training was used to derive 
a mean daily grade for each trainee. 

The mean daily grade data for each group in each stage of instrument flight training 
are summarized in Table 9. For the Stage 1 data, no significant differences between 
groups were found using analysis of variance techniques. A similar analysis of the Stage 2 
grade data did produce a significant difference (F3 gQ = 2.87). A t test following the 
analysis of variance revealed that this difference is between the Low Prior Time and High 
Prior Time Groups, a result that is consistent with the hypothesis that prior aeronautical 
experience would have a facilitative effect on trainee performance. 



1 1 U.S. Army Aviation School (USAAVNS) Regulation 350-16, June 1967. 
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Table 9 

Instructor-Assigned Daily Grades for 
Each Group and Stage of Training 



Group 


Stage 1 


Stage 2 


Mean 


N 


SD 


Mean 


N 


SD 


Experimental 


2.66 


23 


.32 


2.71 


22 


.26 


Control 


2.52 


15 


.37 


2.74 


11 


.32 


Low Time 


2.59 


20 


.37 


2.58 


19 


.33 


High Time 


2.77 


35 


.26 


2.82 


32 


.25 



Check-Ride Grades 

An overall check-ride grade was assigned by the examining pilot at the conclusion of 
each check ride in both stages of training during this research. These grades were assigned 
in accordance with procedures outlined in the Uniform Flight Grading System 1 2 and thus 
provided an independent, although subjective, evaluation of student performance. For the 
purposes of this research, unsatisfactory check rides, which normally do not receive 
numerical grades, were assigned a grade of 65. Table 10 summarizes the check-ride grades 
received by each group on each check ride. Analysis of variance of the data summarized 
in Table 10 failed to show significant differences between groups on either the Stage 1 or 
Stage 2 check rides. 

Table 10 

Check Ride Grades for Each Group 





Stage 1 Check Ride 


Stage 2 Check Hide 


Group 


Mean 


N 


SD 


Mean 


N 


SD 



Experimental 
Control 

Low Prior Time 

(2B12A) 
High Prior Time 

(2B12A) 



79.0 
74.5 

80.0 
82.8 



23 
15 

20 
35 



7.59 
9.58 

6.55 
11.58 



77.0 
79.2 

77.8 
82.2 



23 
11 

13 
32 



8.43 
8.78 

7.76 
7.06 



DISCUSSION 

The primary objective of this initial part of the research was to determine the 
suitability of the new device as a possible replacement for existing obsolete equipment. 
The central question posed by the objective was, "Does training in the new device 
transfer to the aircraft in such a way that device-trained students outperform (in the 
aircraft) non-device-trained students?" The results obtained indicate a qualified affirma- 
tive response to that question. 

Device-trained students, especially early in instrument training, tended to outperform 
students who did not receive device training. During early periods of training the 
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12 USAAVNS Regulation, op. cit 



device-trained students had a lower attrition rate, and they were more likely to be 
above-average students according to their instructors' ratings. On the Stage 1 check ride 
during the fifth week of training, their performance on procedural tasks tended to be 
superior when evaluated subjectively by the check pilots as well as when scored objec- 
tively from photographic records. These findings are consonant with other aviation 
training research where early benefits tend to become attenuated in later stages of 
training (6, 7, 8). 

In this research, a number of factors are believed to have had some bearing on the 
results. First, the training program used with the new device, except for the initial 
training periods, was a modification of the existing synthetic POI and did not take full 
advantage of the new device's unique characteristics. Indeed, the results suggest that the 
device training received prior to actual aircraft instruction was of relatively more value 
than device training that was concurrent with flight training. Also, since the early device 
training was designed to take advantage of the new device's twin-engine configuration and 
later device instruction was not, the attenuation of early benefits may have been due to 
training program content rather than the device itself. 

On the other hand, there is insufficient objective evidence about trainee performance 
in the later stages of training to warrant firm conclusions about the device's contribution 
to this stage of training. Check-ride grades, for instance, are subjective in nature and have 
been shown in most cases to be of questionable reliability (9). Even though in the present 
case the check pilots were from an independent evaluation section, considerable varia- 
bility in the application of the evaluation standards employed by the check pilots was 
known to exist. Time to check ride, while not a subjective measure, is nevertheless 
influenced by administrative constraints and schedule requirements that have little to do 
with trainee proficiency. (Although the training system does not preclude proficiency- 
based advancement (PBA) in training, U.S. Title 10 requirements in effect at the time this 
research was conducted and flight contractor commitments severely limited application of 
the PBA technique in practice.) 

It should be noted that the 31% attrition rate for the Control Group at the Stage 1 
check-ride level presumably comparatively improved the range cf talent for the Control 
Group over that of the Experimental Group by eliminating more of the Control Group 
students who were less capable. In contrast, the device training received by the Experi- 
mental Group may have allowed some trainees to complete the program who otherwise 
would not. Attrition in the advanced stages of flight training is very expensive. Synthetic 
training, to the extent that it reduces attrition, must be considered as making a 
substantial contribution to the cost effectiveness of the overall training system. 

It must also be recognized that the significant differences in attrition rates between 
the Control Group and the two 2B12A groups suggest that training in existing equipment 
is also making a contribution. The training program then being used with the 2B12A had 
been in use by USAAVNS for some time and could be considered as USAAVNS' 
optimized program for that piece of equipment. As noted previously, the optimum 
training program for the GAT-2 was yet to be developed. Nevertheless, students trained 
in the new device with a training program developed for the 2B12A fared as well as those 
trained in the 2B12A with the USAAVNS optimized training program. 

There has been no evidence presented in this research that synthetic training in the 
new device is more effective than similar training in existing equipment. Indeed, a direct 
comparison of the relative contribution of the two devices was not part of the objective 
of this project. Nevertheless, on the basis of its greater overall task similarity to that 
required in the training aircraft, one would predict greater transfer from the new 
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device (2). 1 3 Further, the twin-engine configuration of the GAT- 2 allows the trainee to 
practice several procedures that cannot be simulated in the 2B12A. That such procedural 
training was effective can be seen in the results from the flight instructor questionnaires 
administered at the end of the first week of flight training and also in the check pilot 
ratings of single-engine procedures on the Stage 1 check ride. Nevertheless, a number of 
device characteristics were noted th'at were not considered to have contributed to the 
transfer of training to the T-42 aircraft reported in this section, and some of these 
characteristics may have caused interference with skill development that led to negative 
transfer of training. 

The most likely causes of such effects were found to be (a) lack of instrument/ 
control response uniformity between devices (e.g., major discrepancies were noted 
between indicated airspeed values under supposedly identical simulated, flight conditions); 
(b)the elevator trim in the device did not operate in a manner compatible with elevator 
trim in an aircraft; and (c) the "slop" in the pitch control in the device results in vertical 
speed rates that are excessively (and unrealistically) difficult to stabilize. 

At the conclusion of the research described, an interim report based upon analyses 
of the data contained in this section was delivered to the Aviation School. On the basis 
of the information developed in this research, including the deficiencies just noted, the 
Aviation School prepared and submitted a Draft Proposed Training Device Requirement 
(DPTDR) 1 4 which, if approved, would provide equipment even more suitable for use in 
the Army's fixed wing instrument training program. 
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°The task commonality analysis (TCA) procedures described in Caro (2) were under development 
at the time the device discussed here was under development. The relevant TCA procedures for 
identification of aircraft equipment features were applied to the training aircraft, and the results were 
provided the device manufacturer for his guidance in assuring cost-effective correspondence between the 
two items of equipment. It is believed that the use of TCA procedures in this manner contributed to the 
positive results of this study. 

1 4 Draft Proposed Training Device Requirement "(DPTDR) for a Mufti-Engine Fixed Wing 
Instrument Trainer, 7 April 1970. This action eventually led to the procurement by the Army of 
Device 2B27A. 
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Study Two 



DEVELOPMENT AND VALIDATION TESTING OF AN 
OPTIMUM TRAINING PROGRAM FOR THE NEW DEVICE 15 



RESEARCH OBJECTIVE 

The work described in this section was undertaken immediately following that 
described in Study One. The same research equipment was used, that is, the T-42 aircraft 
and the new device previously described. The need for the research described in this 
section was based upon the assumption, confirmed subjectively by the impressions of the 
research staff during Study One, that a training program that is optimum for one training 
device cannot be optimum for a different device. 

Ideally, it would have been more appropriate to develop an optimum training 
program for the new device before the first part of this research was conducted. Because 
of the Aviation School's desire to generate, as quickly as possible, empirical data on 
which to base a decision concerning the procurement of additional devices, the two-part 
research effort described in this report was undertaken. The decision to procure addi- 
tional devices provided a basis for justification of the research reported here. 

The objective of the second part of this research effort was to develop and validate 
a training program that would be optimum for use with the new training device. 



METHOD 

The procedures involved in the development of the new training program were 
developmental rather than experimental, and involved application of the professional 
judgment of the HumRRO research staff. Some of the concepts and training rationale 
included in the program were taken directly from other pilot training programs (including 
some at the Army Aviation School), while others evolved from concepts found in the 
training technology and human learning literature. During the evolutionary process, 
typical trainees were used to evaluate several methods of instruction and of shaping 
trainee behavior through augmented feedback techniques. Some of these methods and 
techniques were either dropped from further consideration or were extensively modified 
before the total program was made available to the Army. 

Cost effectiveness was the single guiding principle used in devising the new training 
program and each of its elements. For training, cost effectiveness translates to achieving 
given training requirements with the minimal feasible cost in dollars. Toward this end, a 
number of considerations of modern training technology were applied in combination. 
Most notable of these considerations were the following: 

(1) Organization of the training program around a functional context, that is, 
around sets of meaningful, purposeful, mission modules, and teaching training content in 
the context of the mission-oriented purpose it supports. 
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(2) Individualization of training, that is, adapting the pace and redundancy in 
training to the rate of learning of each student and advancing a student to the next set of 
instructional content only after he has demonstrated mastery of an earlier set, 

(3) Sequencing of instruction, that is, arranging the order of instructional 
content so that there is assurance students have been taught (and ha v e mastered) 
prerequisite knowledges and skills before training in a new set is undertaken, 

(4) Minimizing of equipment cost, that is, to the extent that is efficient, 
substituting training in devices or other less expensive equipment for the much more 
expensive training conducted in aircraft, 

(5) Avoidance of over-training, that is, assuring that training time is restricted 
to that needed to bring a trainee to the required level of proficiency and no more, 

(6) Efficient utilization of personnel resources, that is, each instructor should 
be optimally qualified for his task, should be provided with the tools he may require for 
efficient use of his time and talents, and should have clearly stated and measurable 
instructional objectives to attain, 

(7) Use of incentive awards. 

The detailed development of the training program was guided by these principles. In 
addition, the staff engaged in the program development possessed knowledge of numerous 
aviator and other training studies, pilot training practices of other organizations, and the 
Army's training requirements, and detailed familiarity with resources available. 

An attempt to describe the detailed evolution of each individual feature of the 
training program that resulted from this developmental process would not be fruitful. A 
basis for each of these features, either in the training-related literature or in the practices 
of other training organizations, will be recognized by the reader. Instead, this part of the 
report describes the principal features of the program, its initial use by the Army, and its 
training efficiency when con* oared to the program it was developed to replace, 

PROGRAM OBJECTIVES AND LIMITATIONS 

The pilot performance objectives of its twin-engine transition and instrument flight 
training have been stated clearly by the U.S. Army Aviation School in various documents 
related to the course. These objectives, which require course graduates to operate the 
twin-engine T-42 aircraft in accordance with FAA and Army visual and instrument flight 
regulations, were adopted, without change, as the objectives for the new training 
program. Graduates from the new program were to be indistinguishable, as far as flight 
performance is concerned, from graduates of the existing program. 

The intent of the new program, therefore, was to produce pilot graduates who met 
these performance objectives at less overall cost than was being incurred in the existing 
program. Costs can be reduced in flight training programs principally through reductions 
in aircraft operating time. Thus, a reduction in the number of aircraft flight hours 
required to attain the stated performance objectives is the principal means through which 
reduced costs are to be realized. 

Consideration was given during development of the new program to the relative 
costs of all its features to assure that savings resulting from reduced flight-hour costs were 
not offset by costs incurred as a result of the introduction of other training practices. To 
reduce the risk of introducing undesired costs elsewhere, the new course made use of 
only those resources already available at the Aviation School (excluding the new device 
itself) and required the same calendar time for its conduct as did the existing program. 
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PROGRAM SCHEDULE 



The new program is divided into two segments. During the first segment, all flight 
training that can be adapted is conducted in the new device. The aircraft is not used. In 
addition, all required academic training is accomplished during the same period. The 
device and academic training are accomplished on a proficiency basis. Approximately 
three weeks of five working days each are devoted to device and academic training. 
Training in the device, including associated briefing and administrative time, occupies 
approximately four hours each workday, and academic training occupies about an 
additional three hours per day. During the last few periods of academic training, a paper 
procedures trainer (3^ 10) for the T-42 aircraft is used to aid trainees in their transition 
from the new device to the training aircraft. No other training activities are scheduled 
during these first three weeks. 

All scheduled training during the remaining five weeks is conducted in the aircraft, 
although the new device may be used for training at the option of the flight instructor. 
The aircraft training concentrates upon two activities: (a) the conduct of training that is 
required, but that cannot be conducted in the simulated instrument flight rules (IFR) 
environment of the new device, such as landings; and (b) the refinement of skills 
previously trained in the device. 

When, in the opinion of the flight instructors, the trainee is prepared for the check 
ride, which the Aviation School requires of trainees in the existing program, those rides 
are administered by a check pilot. When the trainee successfully passes the end-of-training 
check ride, no further twin-engine transition and instrument flight training is administered 
to him. In the event this occurs prior to the scheduled completion of the course, he is 
excused from further nonessential military duties until the next phase of his training is 
scheduled to begin. 

The new program is designed for administration on an individual basis with advance- 
ment based solely upon student proficiency, that is, the criteria of training are based 
upon attained trainee performance. There is no requirement in the new program that any 
minimum amount of flight training, as measured by hours spent in the device or hours 
logged in an aircraft, be administered to each trainee. To facilitate personnel and 
equipment scheduling, the hours estimated to be required for each type of training, 
indicated in Table 11, are provided. These numbtrs do not represent hour requirements, 
however. Course completion is based solely upon attained proficiency rather than hours 
of exposure to a training environment. For comparison purposes, the hours scheduled in 
the existing pre gram also are presented in Table 11. 



Table 1 1 

Approximate Training Hours Required 





Time (Hours) 


Program 


Aircraft 


Device 


Classroom 


New Program 


35 


25 


45 


Existing Program 


60 


21 


90 



PRINCIPAL PROGRAM FEATURES 

The new training program has been made available to the Aviation School. The 
present description of it is significantly less detailed than the documentation provided the 
Aviation School, since a fully detailed exposition of the program would be of little 
interest to personnel not involved in comparable training. The intent of this description is 
to convey to the reader the general nature of the new program rather than the details 
necessary for its administration. 

Thus, attention is concentrated upon those principal features that distinguish it from 
the existing twin-engine transition and instrument training program of the Aviation 
School and comparable programs of other training organizations. These features are 
described in greater detail in Appendix D. 

The new program incorporated four administrative innovations. First, the instructor 
pilot was given full responsibility for training each student assigned to him; the instructor 
pilot thus became a training manager. Second, a diagnostic progress check ride was 
introduced, to help assure that » student would receive the end-of-course check ride as 
early as possible instead of after a fixed number of flight hours. Third, an incentive 
program was instituted that rewarded student and instructor alike with free time when 
training requirements were met in less than the prescribed course duration. Fourth, 
several procedures were introduced in addition to the progress check ride that provided 
more feedback to the instructor about the progress of his students. 

Innovations in the training content of the new program included provision for device 
training in every procedure and maneuver that could be performed in both the device and 
the aircraft. Thus, the emphasis in aircraft training was upon activities that could be 
practiced only in the aircraft. Another important feature was the capability of the device 
to provide crew training, which also permitted the use of peer instructional techniques. 

AH training was carried out in a functional context rather than in isolated unrealistic 
segments. A substantial economy resulted from the use of a low-cost T-42 paper trainer 
(develo ped by HumRRO) to practice T-42 checklist procedures. Finally, a major change 
was the improved continuity of training achieved by giving one instructor complete 
responsibility for all of the training (synthetic, inflight, and academic) received by each 
student assigned to him. 

Other distinguishing aspects of the new program included (a) individualizing instruc- 
tion through proficiency advancement techniques, (b) using objective measurement to 
determine criterion performance and to pace training, and (c) requiring trainees to be able 
to describe orally all relevant aspects of a given maneuver before they attempt to perform 
that maneuver. 



INITIAL TRAINING PROGRAM USE 

INSTRUCTOR TRAINING 

The new training program has a number of features that are unfamiliar to most 
instructor pilots. For example, it requires that they be familiar with the new device, the 
use of special recording forms, and functional context training. Therefore, before the new 
program could be made available, it was necessary to train flight instructors to use it. 

Two Army officers who were qualified to conduct flight instruction in the existing 
twin-engine transition and instrument training program were designated by the Aviation 
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School to be trained by HumRRO to conduct the new training program.'" Upon 
completion of their training, they were to train other Aviation School personnel in the 
conduct and administration of the program. The two pilots had completed an Aviation 
School Methods of Instruction (MOI) course, the principal purpose of which was to 
assure their standardized performance in the T-42. In addition, both instructor pilots had 
taught twin-engine transition and instrument training courses. Similar qualifications (i.e., 
instructor pilot qualification and aircraft performance standardization) would be required 
of all instructors prior to their training specifically in the conduct of the new training 
program. 

The instructor training for the new program consisted of two phases. The first 
phase, which lasted two weeks, was devoted to standardizing instructor performance in 
the new device, acquainting them with the programed textbooks that the new students 
would be using, and instructing them in the conduct of the new program. The latter 
consisted of instruction in behavioral change techniques and human learning theory, as 
well as how to use Maneuver Performance Records (MPRs) and other course features. 
Specific attention was directed to the modification of existing concepts of the role of the 
flight instructor in relation to the student and the importance of performance criteria 
rather than experience for training. 

The second phase of instruct training lasted eight weeks, a time period that 
corresponded to the scheduled length of the Army's existing twin-engine transition and 
instrument training program. During this phase, each flight instructor was assigned two 
trainees and administered the new prorram to them under the supervision of the 
HumRRC research staff. Training in the device and study of assigned academic material 
occupied the first three weeks of the phase, and training in the aircraft occupied much of 
the remaining time. All student training by these instructors proceeded in accordance 
with the previous description of the new program. 

TRAINING PROGRAM EFFECTIVENESS 

The new program's initial administration by two instructors trained by HumRRO 
conformed very closely to the anticipated hour requirements indicated in Table 11. The 
actual aircraft, device, and classroom hours required for each trainee to achieve the 
performance requirements of twin-engine transition and instrument training are indicated 
in Table 12. The aircraft hour requirement is broken down to indicate the distribution of 
that time by type of training, that is, IFR (actual instrument and hooded flight), visual 
flight rules (VFR) day, and VFR night. 

The students tended to require less device time to reach the performance criteria 
established for that training than had been anticipated. This may be attributed in part to 
a possible "Hawthorne effect," 1 ' since the instructors as well as the trainees expressed 
keen interest in the newness of the device and its far greater apparent training capability 
than was the case with other training equipment. The extent of such a Hawthorne effect, 
if in fact it did occur, cannot be determined on the basis of the initial administration of 

lh Since only two of the new devices had been procured by the Army at tho timo of the training 
reported here, the use of more than two instructors during the initial program administration would 
have been inconvenient. The training of the two flight instructors was under\aken because of the intent 
to convey the necessary skills to Army personnel who then would undertake the training oTan adequate 
number of other personnel. This approach is consistent with HumRRO s contract with the Army to 
engage in training-related research and development activities rather than to undertake directly the 
administration of training programs. 

1 7 A reaction to a change rather than to the effects of the change. 



Table 12 



Distribution of Training Time 
(Hours) 





Device 
Time 3 


Aircraft Time 


Classroom 
Time 


Trainee 


IFR/Hood 


V!=R (Day) 


VFR (Night) 


Total 


1 


19:00 


27:45 


5:10 


1:00 


33:55 


39:00 


2 


19:30 


27:55 


5:10 


1:00 


34:05 


39:00 


3 


21:00 


27:35 


5:55 


1:00 


34:30 


39:00 


4 


21:00 


27:15 


6:40 


1:15 


35:10 


39:00 



^ime during which the trainee occupied the pilot seat only. This figure does not include the time in 
which he received training related to copilot functions or observed the training being administered to another 
student. 



the program. It is believed unlikely, however, that such an effect could invalidate the 
training concepts underlying the new program. 

One trainee required slightly more than the 35 flight hours anticipated in Table 11. 
That particular trainee was grounded for medical reasons the day before he had been 
scheduled by hir flight instructor to be administered an end-of -course performance check 
ride. Upon return to flight status, he was given an additional two-hour period of flight 
instruction before being rescheduled to take a check ride. 

It cannot be expected that all students will complete the new course within the 
35-hour estimate. It is, after all, a course that is administered on an individual, 
proficiency-based advancement, and not all trainees are equally adept at acquiring the 
necessary skills. It can reasonably be expected that the 35-hour estimate will be exceeded 
with the new training program about as frequently as the 60-hour limit of the existing 
program has been exceeded in the past. 

Two check rides (as well as the diagnostic progress rides described earlier) were 
administered to each of the trainees by a check pilot from the Standards Section of the 
Aviation School's Department of Advanced Fixed Wing Training. It was these inde- 
pendently administered check rides that determined whether the trainees had attained the 
course objectives. The check rides were identical in content and standards to check rides 
administered to trainees in the existing course, and in keeping with established Aviation 
School requirements, grades were assigned in accordance with the Uniform Flight Grading 
System described in Study One of this report. The phase flight grades of the students 
involved ranged from 82 to 86 without the addition of incentive points for early 
attainment of the course performance requirements. The average phase flight grade for 
students in the existing course is 83.7. The average was 82.2 for the class from which the 
four students were obtained. 



PROGRAM COST 

The relative effectiveness of any new training program that achieves performance 
objectives identical to an existing program can only be measured by some index of 
efficiency. The index most appropriate to the present situation is cost. The program that 
costs less to administer, all factors considered, would have to be judged the more cost 
effective of the two programs under consideration. It was beyond the scope of the 
present research to determine exactly the cost effectiveness of the new training program, 
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although a model for the collection of aviator training cost data had previously been 
developed by HumRRO (11). 

A study was made of the estimated cost of conducting training in the new 
device (3). On the basis of those data and data provided by the Aviation School at the 
time of this study concerning the cost of flying the T-42, it was determined that an hour 
of instruction in the new device, using the newly developed training program, costs 
approximately $40 less than an hour of instruction in the aircraft. Thus, a reduction of 
25 flight training hours per student, that is, from 60 to 35 hours in the aircraft, would 
yield a savings in flight training costs of approximately $1,000 per trainee. Additional 
savings would be realized from reductions in facilities and in training support personnel 
that would result from the lower flight-hour requirement of the new program. 

APPLICATION TO ANOTHER TRAINING PROGRAM 

Student inputs to the O/WOFWAC were suspended in October 1970. Since that 
time, additional fixed wing Army aviators have been obtained through the cross-training 
of rotary wing Army aviators. The Twin-Engine Transition and Instrument Flight Phase 
of the cross-training program is similar to the comparable Phase of the O/WOFWAC. The 
training objectives are identical, and the training aircraft is the same. Therefore, it was 
inferred that the new training device and many of the training techniques employed in 
the new program developed for it could be applied to the cross-training program as well. 
Consequently, the Aviation School requested HumRRO to investigate the possibility of 
modifying the new training program for use in the cross-training program. 

The cross-training program, more formally called the Fixed Wing Qualification 
Course (FWQC), differed from the O/WOFWAC only with respect to the aeronautical 
experience level of course entrants. Since entrants to the FWQC were already rated 
aviators (i.e., helicopter pilots), they typically completed twin-engine transition and 
instrument training in 45 programed flight hours instead of the 50 hours required for 
O/WOFWAC trainees. The program text:*, used in the new program developed during 
Study Two were developed originally for the FWQC. After review of the new program 
and the FWQC training requirements, it was concluded that no significant modification to 
the new program would be needed. 

A demonstration of the efticacy of the new piogiam for the FWQC was conducted 
using the same instructor pilots who participated in the O/WOFWAC program first 
described. The results of this demonstration, as shown it) Table 13, indicated that, using 
the new training program, flight time in the FWQC fould be reduced from 45 to 
approximately 25 flight hours while maintaining the same quality of performance as 
average course graduates. The results of this demonstration confirm those reported in 
Table 12 for initial program use. 

Pending the delivery of additional GAT-2 devices, the Aviation School also has been 
investigating the suitability of the new device-training program combination for use in 
other fixed-wing training programs. The implications for reducing flight training costs 
without lowering the quality of course graduates are impressive, particularly in an era of 
limited resources for training. 

It should be noted that many of the training concepts employed in the new 
O/WOFWAC and FWQC programs described in this report have application to other flight 
training programs. For example, the training program used in the recently completed 
operational suitability test of Device 2B24 (12), the Army's new helicopter simulator, 
drew heavily upon the GAT-2 training program. Training programs that have been 
developed by HumRRO for U.S. Coast Guard aviators also employ many elements of the 
O GAT-2 training program model. 
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Table 13 

Distribution of Training Time in FWQC 
(Hours) 





Device 
Time 3 


Aircraft Time 


Classroom 
Time 


Trainee 


IFR/Hood 


VFR <Day) 


VFR <Night) 


Total 


1 


19:30 


19:15 


4:45 


1:00 


25:00 


39:00 


2 


19:30 


19:00 


5:00 


1:00 


25:00 


39:00 


3 


16:30 


20:00 


4:55 


1:00 


25:55 


39:00 


4 


16:30 


18:45 


5:40 


1:00 


25:25 


39:00 



'Does not include copilot and solo time. 



TRAINING DEVICES AND TRAINING PROGRAMS 



The introduction of new training devices and equipment into an existing training 
program is a common occurrence in all large training organizations. In the case described 
in this report, it happened to have been in an Army pilot training setting. It could have 
been in many other settings: armor, air defense, submarine, automotive repair, public 
education, mental health, and so on. 

In fact, Study One has been replicated many times. Often, the replication has been 
less formal— perhaps without a designated control group or elaborate data-gathering 
procedures, but the general approach and the results have been comparable. The approach 
consists of substituting a new training device for an obsolete or- worn-out one, adjusting 
the training program to fit the new device, and training a few students just to check 
everything out before procuring the new device in quantity. Although there have been 
exceptions, the general procedure in such cases is to gather little or no formal data, 
because the benefits of the new equipment are obvious. Observed increases in training 
efficiency in such cases are attributed to the new training device, and the decision to 
procure mc/e devices, which occurred as a consequence of information gathered during 
Study One, is an empirically justified decision. 

Had a similar procedure been followed in this research— that is, had the effort 
terminated at the conclusion of Study One, and had Study Two not been undertaken— the 
Army would have realized relatively little of the potential benefits to be derived from use 
of the new device. It probably would never have been known that approximately a 40% 
increase in training efficiency could be realized simply by developing a training program 
designed specifically for the new device instead of using an existing program (even with 
adjustments required by some of the features oi the new equipment), which was designed 
for older devices. 

The need for effort corresponding to Study Two should be apparent in almost all 
training programs where new or modified training equipment is introduced. The need is 
greatest, of course, in instances where the older equipment has been in use for an 
extended period of time. Advances in the technology of training during the past decade, 
particularly advances related to the design and use of training devices, almost guarantee 
that device training programs that have not undergone extensive updating in the past few 
years probably are not as effective as they can be. The introduction of new equipment 
usually provides an ideal opportunity to take advantage of new information and tech- 
niques related to training, and efforts to develop optimum training programs for such 
new equipment should be made whenever these opportunities arise. 

It is surprising, therefore, to observe that many training organizations, military and 
civilian alike, will tend to ignore the importance of training program design when new 
equipment is introduced. There are many instances in which literally millions of dollars 
have been invested in the development of training vehicles, devices, and simulators, but 
not one dollar has been devoted to the development of appropriate methods and 
techniques for their use. Instead, the new equipment typically is turned over to training 
personnel whose orientation and experience is limited to existing programs, procedures, 
personnel, and policies, and who are instructed to upset the training schedule and the 
personnel structure as little as possible during the time the new equipment is being 
introduced and tested—and above all, to stick with the training syllabus already approved 
by higher headquarters! Under circumstances such as these, there is little chance that 
Study Two type of program development activities can be undertaken. 

O 

ERIC 

31 



ERIC 



As a consequence, training program designers and administrators tend to rely upon 
the device itself to assure adequate training. Too often, people appear to forget that 
training equipment doesn't train. It is the manner in which training vehicles, devices, and 
simulators are used that yields the benefit. A training program must be developed for 
each situation in which skills are to be developed, and each program must take into 
account the capabilities of the specific equipment used if its potential contribution to 
transfer of training is to be realized. 

It has also long been known that factors other than equipment/device similarity 
impact on transfer of training. For instance, Gagne ( 13) pointed out that transfer of 
training from devices and simulators to operational equipment is a function of factors 
such as training objectives and instructional quality as well as the characteristics of the 
training equipment. Muckler et ai, (14) identified instructional techniques and instructor 
ability as important variables involved in transfer of training. Prophet (JJ stated that 
training devices are only the vehicles for the training programs, and they often are less 
important than are the training device instructor and the organization and content of the 
device training program. 

Probably no one would maintain that the manner in which training equipment is 
used is unimportant. In spite of this, attention to the program of appropriate use of 
training equipment, particularly newly procured training equipment, is seldom sufficient. 
There are many studies conducted to determine whe <sr a new item of training equip- 
ment is useful, but there are few studies conducted to determine the best way to use new 
equipment. When viewed together, the two studies reported here lend support to the 
notion that the training program used with a device is more important, from the transfer 
of training standpoint, than the device itself. 
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Appendix A 
SYNTHETIC TRAINING SYLLABI 





New Device 


Device 2B12A 


Flight Syllabus 






I Period 






Period 


Training Day 


Period 


Subject 


j Time 3 


Period 


Subject 


Time 


- 


1 


Cockpit Procedures 


1:10 










2 


Basic IFR Maneuvers 


1:10 










3 


Review and S.E. 
Procedures 


1:10 










4 


Emergency Pro- 
cedures 


1 :10 










5 


Review 


1:10 








1 u 














\iiu byniriciic iraiiiiiiy/ 














7 


6 


Proficiency IFR 














Maneuvers 


1 :00 


I 


Intro, to trainer 


1 .oil 


Q 

o 


1 


Review and intro. to 

D -is-in*. /DAD 0. 

Haaar (rAn ot 














Abn) 


1 \j J 


Z 


Review • Basic IFR 
Maneuvers 


1 '^n 


9 


8 


Review & intro. to 














VOR/RMI 


1:00 


3 


Review - intro. to 
GCA (PAR & « 
ASR) 


1:30 


10 


9 


VOR/RMI 


1:00 


4 


Review & intro. to 
VOR/RMI 


1:30 


11 


10 


Review VOR/RMI 


1:00 


5 


Review 


1:30 


12 


11 


Review & intro. to 














ADF/RMI 


1:00 


6 


Review 


1:30 


15 


12 


ADF/RMI 


1:00 


7 


Review 


1:30 


17 


13 


Review ADF/RMI 


1:00 


8 


Review 


1:30 


19 


14 


Review & intro. to . 














ILS 


1:00 


9 


Review 


1:30 


20 


15 


ILS 


1:00 


10 


Intro, to ADF/RMI 


1:30 


21 


16 


Review ILS 


1:00 


11 


Review 


1:30 


22 


17 


Review 


1:00 


12 


Intro, to ILS 


1:30 


23 


18 


Enroute navigation 


1:00 


13 


Review ILS 


1:30 


24 


19 


Review 


1:00 


14 


tznroute navigation 


1:30 b 


25 


20 


Review 


1:00 


15 


Review 


1:30 


26 


21 


Review 


1:00 


16 


Review 


1:30 


Total 






21:50 






24:00 



Period time in the new device is pilot time only; in addition, trainees received approximately 15 hours of copilot/ 
observer training. 

Completion of 21 :00 hours represents the minimum synthetic trainer requirement. 
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SYNOPSES OF STANDARDIZED CHECK RIDES 

Stage 1 Check Ride 

1. Take-off, VFR. 

2. GCA. 

a. Hood ON . (Established in climb at instructor's discretion.) 

b. Student : contacts Cairns approach control for a GCA with missed 
approach. 

c. At the command, "Contact final controller, 

Camera and recorder ON . 

3. Missed approach. After student is established in climb: 

a. Hood OFF . 

b. Recorder OFF . 

c. Climb to 4500 feet. 

d. Slow cruise. 

e. Cardinal heading. 

4. Steep turns, VFR. 

a. Right turn , 45° bank , 360° . 

b. Left turn, 45° bank, 360° . 

5. Clearing turns (not graded). 

6. Stalls, straight ahead, VFR. 

a. Gear - down. 

b. Flaps - down (28°). 

c. Power OFF . 

7. Unusual attitudes, IFR (slow cruise). 

a. Student : puts hood on. 

b. Descending turns to the left, 150 kts, recover to slow cruise. 

c. Climbing turn to right, 80 kts, recover to slow cruise. 

d. Remove hood. 

8. Single engine, VFR (slow cruise). 

a. Cut left engine. 

b. Clean-up, full feather. 

c. Restart. 

d. Go to normal cruise. 

9. Gear extension (manual). 



10. VOR, IFR. DHN. 

a. Puts hood on. 

b. Orientation. 

c. Recorder ON . 

d. Issue clearance: "Army is cleared to the Dothan VOR via the 210° 
radial, maintain 3500 feet. Upon arrival hold southeast on the 154 radial, 
left turns, expect approach clearance at 

e. After second circuit (third time over the fix), clear the student for a Dothan 
VOR 1 approach to report procedure turn. 

f. Make VOR approach. 

11. Missed approach: start. 

a. After climb is established, remove hood. 

12. Landing, VFR. 

a. Make normal pattern entry on downwind. 

b. On downwind, cut left engine after gear is down. 4 

c. Land. 

13. Take-off, VFR. 

a. Return to Cairns AAF. 

b. Landing 

c. Camera and recorder OFF. 



Stage 2 Check Ride (IFR) 



Flight plan filed to Dothan with Cairns AAF listed as an alternate. 

1. Clearance receipt and readback. 

a. RECORDER ON. 

b. Clearance: "Army is cleared to the Echo intersection via Cairns 

021 radial to Skipperville, V7 Echo. Maintain 3000. Depart runway . 

After take-off left/right turn heading to intercept the Cairns 021 

radial. Departure frequency , squawk low just before 

departure." 

c. Student: reads clearance back. 

2. Instrument take-off. 

a. Camera ON. 

b. Line up. 

c. Take-off. 

d. Student: reports time off . N 

* — : 7 ;*r> , 4. x «i (Direction) . ~ . 

e. Instructor: Radar contact miles _of Cairns, 

report level at 3000. M 

f. Student : rogers instruction. 

3. Hand off. 

a. Student : reports level at 3000. 

b. Instructor: "Roger, radar service terminated miles southwest of 

Skipperville, contact Cairms approach control frequency 118.4 now. 1 ' 

c. Student : rogers instructions, switches to 118.4 and reports: "Cairns approach 
control Army estimating Skipperville at , 3000. " 

d. Instructor: "Roger, Army , hold northwest at Echo on V7, expect 

further clearance at 

e. Student: rogers instructions. 



At Echo (hold 2 circuits using CD Is), 

a. Student: enters holding pattern (slow cruise). 

b. Student : reports reaching holding fix. 

c. Instructor : fails light engine. 

NOTE: Give the engine back at instructor discretion. 

d. Instructor: issues following clearance during second holding pattern: "Army 

is cleared to the Dothan VOR, from over Echo, via V7, maintain 

3000', hold southeast of Dothan on the 154 radial, left turns, 1 -minute pattern, 
expect approach clearance at 

Departing Echo. 

a. Student : reports departing Echo. 

b. Instructor: take away GDIs. 

Dothan VOR. 

a. Student : enters holding pattern (goes to slow cruise). 

b. Student: reports reaching Dothan VOR. 

c. Instructor: does not answer, simulates lost communication. 

d. Student: follows lost communication procedures. Departs Dothan VOR, 
outbound for approach, at expected approach time. 

e. Instructor : re-establishes communications on missed approach. 

Missed approach. 

a. Student : at the appropriate time, initiates missed approach, reports, and 
requests clearance to Cairns AAF, his alternate. 

b. Instructor : issues clearance and tells the student to contact Cairns approach 

control on frequency , and request an ILS approach with radar vector 

to the outer marker. 

c. Instructor: as soon as the student is established. on course and altitude, instructor 
fails the RMI. Give the RMI back prior to turn toward intercepting localizer. 

Outer marker. 

a. Instructor : fails left engine. 

b. Student : executes an ILS approach to circling or straight in landing. 

c. Instructor: after touchdown, CAMERA and RECORDER OFF. 



Student: 
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CHECKLIST RECORDS OF FLIGHT PERFORMANCE 

Check Pilot's Flight Checklist (Stage 1) 

Date: 



Check Pilot:. 
Class No.: 



A/C No.:. 



Approximate Time of Take-off: 



Type Check Ride:. 
Landing: 



Weather at Cairns Field: 

At beginning of check flight: 

Wind direction; ; velocity. 

Turbulence: none ; light 



knots 

moderate. 



At end of check flight: 

Wind direction: ; velocity knots 

Turbulence: none ; light ; moderate. 



Student's Tension: 

None apparent. 



moderately tense. 



very tense. 



Check pilot comments on photographed portion of the flight: (Indicate any deviations from 
standardized flight profile) 
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Check pilot rating of student performance. Circle the appropriate response. (Omit any item 
not included in the check ride.) 



Take-off (VFR) 






Landing (VFR) 








line up 


left 


proper 


right 


natfprn pntrv * 

uai lcl 11 ci i Li y . 








power application 




proper 


improper 


pntrv noint 




proper 


improper 


heading (runway) 


left 


proper 


right 


altitnHp 

ul 1/11/ \X\JL \Z 


high 


proper 


low 


rotation 


early 


proper 


late 


ail bpt;<rLl 




proper 


improper 


gear up 


early 


proper 


late 


HnwnwinH Ipct* 

UU VV 11 VV 111U * £5 . 








heading control 








Hictani»P r»iit 

\JL lo Ldl 1 C \J U L 




proper 


improper 


(climb-out) 




proper 


improper 


HpqH ino" 

LlCdtl lUfc 




proper 


improper 


power reduction: 












proper 


improper 


throttles 




proper 


improper 


flaps 




proper 


improper 


props 




proper 


improper 


mrknit rhprk 




proper 


improper 


sequence 




proper 


improper 


alt rnntynl 

ul L . t-Ull Li. KJ 1 




proper 


improper 










airQnppH 

a LL OULLU 




proper 


improper 


Recovery From Unusual Attitudes (IFR) 


base lesf 
















turn 


early 


proper 


late 


power 




proper 


improper 


heading 




proper 


improper 


pitch 




proper 


improper 


airsrippd 


nign 


pr upci 


low 


roll 




proper 


improper 


flaps 




proper 


improper 


altitude control 




proper 


improper 


descent 




proper 


improper 


sequence 




proper 


improper 


final: 
















turn 


early 


proper 


late 


Single Engine Procedures 






heading 




proper 


improper 










airspeed 




proper 


improper 


heading control 




proper 


improper 


descent 




proper 


improper 


altitude control 




proper 


improper 


landing: 








power control 




proper 


improper 


touchdown 


early 


proper 


late 


clean up 




proper 


improper 


attitude 




proper 


improper 


restart 




proper 


improper 


heading 


left 


proper 


right 










brake appl. 




proper 


improper 


Manual Landing Gear 








cockpit 








Extension 




proper 


improper 


cleanup 




proper 


improper 


Comments: 
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Student: 



Check Pilot's Flight Checklist (Stage 2) 



Date: 



Check Pilot:. 
Class No.: 



A/C No.:. 



Approximate Time of Take-off: _ 



Type Check Ride:. 
Landing: 



Weather at Cairns Field: 

At beginning of check flight: 

Wind direction: ; velocity knots 

Turbulence: none ; light ; moderate. 



At end of check flight: 

Wind direction: 

Turbulence: none 



velocity. 
; light_ 



.knots 



_; moderate. 



Student's Tension: 

None apparent^ 



moderately tense_ 



.; very tense 



Check pilot comments on photographed portion of the flight: (Indicate any deviations from 
standardized flight profile) 
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Check pilot rating of student performance. Circle the appropriate response (omit any item not 
included in the check ride). 



Take-off (ITO) 
line up 

power application 
heading (runway) 
rotation 
gear up 

heading (climbout) 
power reduction: 

throttles 

props 

sequence 

Track Interception 

turned to intended 

heading 
rolled out on 

course 

Holding Pattern 

entry 
timing 
altitude 
track 



left 

early 

early 



proper 
proper 
proper 
proper 
proper 
proper 

proper 
proper 
proper 



proper 



proper 



proper 
proper 
proper 
proper 



improper 

improper 

right 

late 

late 

improper 

improper 
improper 
improper 



improper 
improper 



improper 
improper 
improper 
improper 



Missed Approach 

time 

report 

procedures 

Communication 

tuning radios 
reporting 
lost comm. 
procedures 

Partial Panel 

heading control 
bank control 
airspeed control 
altitude control 



proper 
proper 
proper 



proper 
proper 



proper 
proper 
proper 
proper 



Single Engine P rocedures 

heading control proper 

altitude control proper 

power control proper 

clean-up proper 



improper 
improper 
improper 



improper 
improper 



proper improper 



improper 
improper 
improper 
improper 



improper 
improper 
improper 
improper 



Comments: . 
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Appendix D 



A DESCRIPTION OF THE PRINCIPAL FEATURES OF 
THE NEW TRAINING PROGRAM 

ADMINISTRATIVE FEATURES 

Some of the distinguishing features of the new program are administrative in nature. 
The most significant of these is the concentration of responsibility for all of a student's 
training in the hands of his flight instructor. Traditionally, in undergraduate training 
programs, the flight instructor has been responsible for only that portion of a student's 
training that takes place in the aircraft, while a device instructor is responsible for device 
training, and several classroom instructors share responsibility for other portions of 
training. Presumably, this divided responsibility is compensated for by a superior manage- 
ment effort or higher instructing skills. In practice, however, flight instructors often 
complain that device instructors are teaching the wrong thing or that classroom instruc- 
tors do not present necessary information until some time after it is needed in the air. 

In the new program, by contrast, the flight instructor is the instructor. He conducts 
or oversees all training received by his students. The resources he needs— an aircraft, a 
training device, programed textbooks, and personnel to assist him when required—are 
made available. 

During the initial three weeko of the course, the flight instructor conducts carefully 
structured training in the device. During this period, he also assigns to his trainees 
programed texts 1 covering all the information that they will require, and he does this on 
a schedule that assures that they have covered the material prior to needing it in the 
trainer. An assistant, who is familiar with the programed texts, is available during this 
period to help the students should they have difficulty with the material, but the effort 
of the assistant is supervised directly by the flight instructor. It should be noted that 
much of the training provided by the flight instructor in the device could not be achieved 
by a nonrated device operator. 

After each trainee has completed all the required programed texts and has developed 
all the specified skills in the device, the locus of instruction is shifted to the aircraft. This 
occurs at about the beginning of the fourth week of training. The remaining scheduled 
training takes place in the aircraft. However, should the instructor feel that additional 
practice in the device would help the trainee overcome a particular problem, remedial 
device training may be prescribed. 

Training in the aircraft is expensive, and the instructor's effort is directed toward 
minimizing requirements for it. His responsibility, so far as inflight training is concerned, 
is to prepare his student for the final check ride to be administered by the Aviation 
School's flight check personnel. It is emphasized to the instructor that building up 
student flight time is not the goal of the new program; rather, it is the building of flight 
skills to the specified performance level. Therefore, on occasions when the student 
appears unresponsive during flight instruction— as students occasionally do— the instructor 



Programed textbooks which contained all the information required to be taught in the new program 
were available from other courses at the Aviation School. 
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is encouraged to terminate the period. To continue to fly a student under these 
circumstances only adds expense to an already mostly program. Thus, there is no 
requirement to log any specific amount o? ? aircraft time per student per day. 

To encourage the flight instructor to prepare his students in a minimum amount of 
time, a progress ride identical in content io che end-of-course check ride is administered 
at an hour level between two-thirds and ihree-fourths of the total projected flight time 
that conduct of the course should take, that is, at about the 25-hour level. Students 
passing this progress ride will have demonstrated end-of-course proficiency and can be 
graduated from the phase at that point. One purpose of the progress ride is to discourage 
flight instructors from holding back a student beyond the time when he could pass a 
check ride just so that he might attain a higher grade. Such a practice adds unnecessary 
cost to training. 

The primary purpose of the progress ride, however, is diagnostic. It is intended to 
identify those skills upon which the instructor should concentrate in order to prepare his 
student for the end-of-oourse check ride as soon as possible. An important part of the 
progress check ride, then, must be a provision for detailed feedback to the instructor 
involved. This feedback is accomplished by having the check pilot debrief the instructor 
in detail on trainee performance. This debriefing occurs right after the progress ride itself. 

Attaining high grades has certain useful motivational value in flight training as does 
building up more flight time. It could be expected that a course designed to lower 
student grades and reduce the amount of flight time he can log might be poorly received. 
The new program includes an incentive award system designed to overcome these 
problems. The award system is performance oriented rather than schedule oriented and 
has two elements: 

First, a student is awarded a check-ride grade by his check pilot as is 
customary. Added to that check-ride grade is a number of points determined by the hour 
level at which he completes training— the lower the total flight time required to complete 
training the greater the number ot points awarded. For example, with a "schedul'J" 
course length of 35 hours, a trainee who passes the course check ride in 30 hours with a 
check-ride grade of 75 would be credited with an additional five points, one point for 
each hour saved. His adjusted check-ride grade would be 80, a grade that would more 
accurately reflect his ability to develop skill as a pilot. 

Second, free time is used as an incentive. Other HumRRO research has shown 
that Army trainees attach considerable value to free time and that it can be used to 
motivate them (15). In the program under review here, a trainee who passes the required 
check ride in less thai- the allotted eight weeks is given time off from all nonessential 
military duties for the remainder of the course. WhetMrffTftriQees assigned to a particular 
instructor complete the course requirements, then the instructorS^also awarded time off 
as an incentive to expedite this training and reduce expensive flight th^ie. 

Thus, there are four principal administrative aspects of the hew program that 
distinguish it from traditional practices: (a) the assignment of full responsibility for 
training to the flight instructor; (b) a diagnostic progress ride at a set point in training; 
(c) feedback to the instructor pilot; and (d) an incentive award system. 

TRAINING CONTENT FEATURES 

The new device is by no means a T-42 simulator. As noted in Study One, it is 
particularly weak with respect to handling qualities, and it is very difficult to trim. 
Furth t, the cockpit controls and the instrument panel are notably different from those 
of the training aircraft. Nevertheless, the similarity between the T-42 and the new device 
q is much greater than that between the T-42 and any other pilot training device in the 
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Army inventory. The significance of these similarities is that they permit the device to be 
treated as an aircraft rather than as an item of training equipment. The simulation 
provided in the new device is sufficiently complete that it is a plausible representative of 
an unspecified light twin-engine aircraft, such as the T-42. A pilot able to fly one such 
aircraft, presumably, could make the transition to another with minimum difficulty* 

During the first three weeks of student training, when the device and programed 
textbooks were being studied, every procedure and maneuver that could be performed in 
both the device and the aircraft were performed in the device. These included all aircraft 
prestart, start, run-up, and shutdown procedures, using checklists prepared to correspond, 
whenever possible, to the T-42 checklists; IFR missions, from instrument take-off through 
approaches and missed approaches; and, to a limited extent, training to cope with inflight 
emergencies. In the case of the latter, for example, a fuel starvation engine failure may be 
simulated, and the single-engine procedures similar to those required in the aircraft can be 
practiced. 

All this training is performed in the context of crew training. Because of dual 
instruments and controls, a copilot seat, and a separate jump seat, two trainees always 
occupied the device. Except during initial training sessions where instructor assistance 
may be required with aircraft control tasks, a second student, seated in the copilot seat, 
performs normal copilot duties under instructor supervision. In addition to permitting 
specialized copilot training, when the instructor is not required in the copilot seat, this 
exposure to the training of a fellow student has been shown, in other research (15, 16), to 
aid pilot trainees, particularly where procedural and communications skills are concerned. 

All training in the course is functional with respect to operational requirements. The 
first training period in the device, for example, is modeled after flight tasks typical of 
instrument flights in the training aircraft. It consists of performance of necessary aircraft 
checks and starting and run-up procedures, an instrument take-off, climb to assigned 
altitude, compliance with radar steers, and initiation of a VHF omnidirectional range 
(VOR) approach. Typical training periods consisted of filing an TFR flight plan and 
executing, in the trainer, the prescribed flight. The widely practiced procedure of 
practicing "basic airwork" during early training periods is not followed in the new 
program. 

Transition to the aircraft is designed to require minimum disruption in training. As 
trainee*, begin to become proficient in the device, they prepare to transition to the T-42. 
Using a very low-cost T-42 paper trainer developed by HumRRO, T-42 checklist pro- 
cedures are practiced to mastery so that minimum aircraft time is lost because of 
unfamiliarity with the aircraft itself. As a result, the student spends less time in executing 
necessary procedures than he would lacking such training. Getting into the air during the 
first aircraft flight period requires no more time than for students who typically have 20 
to 30 hours of experience in the T-42. Transition is further enhanced by the fact that 
students and instructors are already familiar with each other, and the only thing new 
about the situation is that the actual aircraft is now being used. 

The first flight period in the aircraft is largely a repeat of the last flight in the 
device, using a similar IFR clearance. Since each trainee has demonstrated in the device 
that he can plan a flight, file a flight plan, and fly the flight within specified tolerances in 
the device, his task during the first aircraft period is that of doing the same thing in the 
aircraft. In other words, the portion of the new training program conducted in the 
aircraft is primarily that of transitioning to the aircraft and polishing skills already 
acquired in the trainer. 

There are, of course, aircraft control skill requirements that cannot be practiced in 
the device and have to be learned in the aircraft itself. These skills largely relate to 
control of the aircraft by external visual reference. A portion of the flight training in the 
aircraft, therefore, must be devoted to development of these VFR skills. 



The VFR training also is conducted in a functional context. The usual practice of 
scheduling periods of VFR training early in the course in order to check out the trainee in 
the aircraft before instrument training begins is not followed in the new program. 
Instead, the required VFR training is conducted, as occurs in actual practice, as a part of 
a practice IFR cross-country flight. For example, at destination, students fly traffic 
patterns and land the aircraft h. accordance with simulated or actual ATC instructions. 
This is a type of application of the integrated contact-instrument flight training concept 
studied in previous HumRRO research (6). As with all other training in the new program, 
the VFR training in the aircraft is accomplished on a proficiency basis. The instructor is 
required to train to specified performance objectives and there are no hour or experience 
requirements involved. 

Thus, there are four prin^pal training features of the new training program that 
distinguish it from the existing course— features that, for the most part, are made feasible 
by the correspondence between the device and the training aircraft. These training 
features are (a) provision of full mission training in the device, (b) the training of pilot 
and copilot skills within a crew training context, (c) the conduct of all training within a 
functional context, and (d) the continuity of training conducted in the device and in the 
aircraft. 



PROFICIENCY ADVANCEMENT 

There were other distinguishing aspects of the new program, in addition to admin- 
istrative and training content features. One of these, the individualization of instruction 
through proficiency advancement techniques, already has been identified. All training- 
device, academic, and flight— is conducted on a proficiency advancement basis in the ne\v 
course, and there are no requirements that a student receive a stated amount of training 
of any kind at any time. 

Advancement of students from one block of instruction to another is accomplished 
in an objective manner. Maneuver performance records (MPR) were developed to 
facilitate this. The MPRs enabled the flight instructor to attend to each parameter of 
student performance during execution of a maneuver in the device and to record each 
deviation from the required performance objective. Using the MPRs, the instructor 
advances trainees from maneuver to maneuver in ah objective manner that precludes 
unnecessary practice beyond required levels of mastery. An example of an MPR is shown 
in Figure D-l. 

The MPR serves another important function as well. It provides detailed descriptions 
of trainee performance throughout his training in the device. The descriptions are an 
excellent means of providing knowledge of performance to the trainee, since they can be 
(and are) reviewed with him following each maneuver he performs. Specific, detailed 
information is thus available to both the instructor and the student concerning errors in 
student performance, progress over trials, and the changes in performance yet required 
before the necessary performance objective is reached. 

In this new course, trainees are required to demonstrate appropriate skills in a 
progressively functional manner. Before any maneuver is performed by a trainee in the 
device, for example, he is required to describe all relevant aspects of that maneuver in 
detail to his flight instructor. Before the same maneuver is performed in the aircraft, it 
must have been performed without error in the device. The MPRs are of considerable 
value to both the instructor and the student in accomplishing this progressive training. 
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Maneuver Performance Record (Sample) 



Single Engine Procedures 
(Cruise) 



Student 



Sheet No. 



Date 1 
Power (as required) 
Identified dead engine 
Throttle (dead engine) closed 
Prop lever (dead engine) feather 
Gear (as required) 
Flaps (as required) 
Power (as required) 
Trim 

Cowl flaps (as inquired) 

Fuel quantity switch & selectors, 
(as required) 

Alternator (dead engine) off 
Airspeed 102 kts min. 
Altitude (±100') 
Direction (±10°) 



n k % "A 



y 



y 



y 



i/ 



y 



y 



y 



y 



y 



y 



y 



y 



y 



y 



y 



y 



y 



Errors 
Total prior time: 



1st 
- trial 



Z 



Criteria 
reqched 



Figure D-1 
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1 HUMAN FACTORS TEST DtV EGLIN AFB 

1 PERS SUBSYS DIV CREW SUBSYS DRCT AEROSyS DIV WRIGHT-I 

1 DIR USA BD FOR AVN ACCIDENT RSCH FT RUCKER 

1 LIB OEF SUPPLY AGCY CAMERON STA 

1 LIB ARMY WAR COLL 

1 COMDT COMMAND & GEN STAFF COLL ATTN ARCHIVES 

1 DIR OF MIL PSY & LDRSHP USMA 

1 COMDT ARMY AVN SCH ATTN DOI 

1 LIB USA ARMY SCH 

1 COMDT USA TRANS SCH ATTN EA 

1 COMDT ARMED FORCES STAFF COLL 

1 COMDT USA AVN SCH ATTN EA 

1 HQS APG ATTN TECH LIB 

1 COMDT USA C&GS COLL OFC OF CHF OF RESIDENT INSTRUC 

1 COMDT USA CIVIL AFFAIRS SCH ATTN LIB 

1 LIBN USA INF SCH FT BENNING 

1 COMDT USA TRANS SCH ATTN LIB 

1 COMDT USA FLD ARTY SCH ATTN LIB 

1 ENGR SCH LIB FT BELVOIR 

1 ACADEMIC LIB FT HARRISON 

1 DCS PERSONNEL DA ATTN CHF C&S DIV 

1 DIR OF PLNS STUDIES & BUDGET ODCSPER 

1 ASST CHF OF STAFF FOR FORCE DEVEL ATTN CHF TNG DIV 

1 HQ ARMY MAT COMD R&D DIRECTORATE AMCRD-RC 

1 CHF OF PERSNL OPS DA ATTN OPSC 

1 DA OPO ATTN MOS SEC (NEW EQUIP) OPOPMO 

12 ADMIN DDC CAMERON STA 

1 CHF OF R & D ATTN CHF TECH & INDUST LIAISON OFC DA 

1 TNG & DEV DIV ODCSPER, DA 

1 OIR USA RSCH INST FOR BEHAtf & SOC SCI 

1 USA LIB DIV TAGO ATTN ASDIRS 

1 PRES USA AVN TEST BD 

1 CDR USA TRADOC ATTN ATTS-ITR 

15 CG FORCES COMD ATTN AFOP-RE FT MCPHERSON GA 

2 CG TRADOC ATTN LIB 



CBT ARMS TNG BD FT BENNING 

CHF USA AVN HRU FT RUCKER 

CG 2ND ARMORED DIV ATTN DIV AVN OFCR FT HOOD 

CG 4TH ARMORED DIV ATTN DCSOT 

CG 82ND ABN INF DIV ATTN ACOFS G3 

CG XVIII ABN CORPS ATTN ACOFS G3 

CO USA TNG DEVICE AGCY 

DIR ARMY LIB 

CHF OF MIL HISTORY DA ATTN GEN REF BR 

USA GEN EQUIP ATTN TECH LIB FT LEE 

CO USA RESCH OFC DURHAM NC 

BSO. SCI DIV ARD 

RSCH CONTRACTS & GRANTS BR ARI 

TECH LIB PERS 11B BUR OF NAV PERSNL 

DIR PERSNL RSCH DIV BUR OF NAV PERSNL 

TECH LIB BUR OF SHIPS CODE 210L 

CO NAV TNG EQUIP CTR ATTN TECH LIB 

CO US NAV AIR DEV CTR ATTN NADC LIB 

CHF OF NAV RSCH PERSNL & TNG BR (CODE 458) 

DIR US NAV RSCH LAB ATTN CODE 5120 

DIR NAV RSCH LAB ATTN LIB CODE 2D29 (ONRL ) 

CHF OF NAV AIR TNG TNG RSCH DEPT NAV AIR STA 

DIR AEROSPACE CREW EQUIP LAB NADC 

OFCR IN CHARGE US NAV PERS RSCH ACTIVITY SAN DIEGO 

COMDT HQS, US MARINE CORP CODE AOIM 

DIR MARINE CORPS INST ATTN EVAL UNIT 

CHF NAV AIR TECH TNG NAV AIR STAT MEMPHIS 

DIR OPS EVAL GP OFC OF CHF OF NAV OPS 0P03EG 

COMDT PTP USCG HDQ WASH DC 

SUPT USCG ACAD NEW LONDON CONN 

HQ ATC ATTES RANDOLPH AFB TEXAS 

TECH DIR TECH TNG DIV (HRD) AFHRL LOWRY AFB 

DEPT OF THE AF HQS ATTN AFCIN-3D1 

CHF SCI DIV DIRCT SCI & TECHNOLOGY AFRSTA 

FAA DIR OF PLNS ft OPS DCS/P&O 

RSCH & ANALYSIS DIV HQ USAF 

ATC ATXRQ RANDOLPH AFB 

AFHRL/TT ATTN CAPTAIN WAYNE SELLMAN LOWRY AFB 

AIR FORCE MIL TNG CTR XRS LACKLAND AFE 

AFHRL (HRT) WRIGHT-PAT AFB 

HQS ATC DCS/TECH TNG (ATTMS) RANDOLPH AFB 

AF HUMAN RES LAB MANPOWER DEV ATTN COL RATLIFFE 

DIR AIR UNIV LIB ATTN AUL3T-63-253 MAXWELL AFB 

USAF SCH OF AEROSPACE MEDICINE ATTN DOCUMENT SEC 

USAFA DIR OF LIB USAF ACAD 

AFHRL/PE LACKLAND AFB 

TECH TNG CEN (LMTC/OP- I-Ll ) LOWRY AFB 

CO AF HRL (AFSC) ATTN D0J2 

CO HRL BROOKS AFB 

AFHRL (FT) WILLIAMS AFB ARIZ 

CHF MGMT & GEN TNG DIV FAA 

OFC OF INTERNAT'L TNG PLNG & EVAL STAFF AI 

DOT FAA ACQ SEC 

ACQ LIBN ERIC 
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